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It has often been assumed that differences in interviewers 1 behaviors and 
attitudes may be an important factor affecting responses to survey questions; 
and much interviewer training is aimed at standardizing interviewer behavior 
in o^-der to minimize biasing influence* Survey methodo log i sts have been 
studying interviewer effects for several decades (e.g.. Freeman and Butler, 
1976), and the Bureau of the Census has an equally long history of measuring 
the impact of interviewer variability on census statistics (Bailar, 1 976 ; Han- 
son and Harks, 1958) and* more recently* in sample surveys conducted by the 
Bureau (Bailar, Bailey* and Stevens, 1977; Bailey, Moore* and Bailar* 1976) * 
Similarly, in market research it has long been accepted that variation in 
response to survey questions caused by differential interviewer questioning 
techniques or interpretation of responses can be a major source of bias or 
nonsampling error (Boyd and Westphal* 1955* 1 9&5 » 1970; McKenzie* 1977)* Con- 
siderable research is also carried out by survey methodo log i sts demonstrating 
that individual differences of interviewers as welt as interactions of 
respondent and interviewer characteristics produce different responses from 
the same respondents (e,g*, Cahalan* et a1«» 1£)W; Dohrenwend, et al*, I968; 
Freeman and Butler* 1976; Groves and Kahn, 1979; Hyman, 195*4; Kish, I962; 
Stock and Hochctim* 1951; Sudman and Bradburn, 197*+; Sudman* et al** 1977)* 
Although these studies differ substantially in populations sampled, interview- 
ing staffs* types of variables analyzed, and in the particular statistics they 
examined, in combination they suggest that as much as 5 to 10 percent (or even 
more) of the variation in respondent reports of certain attitudes* behaviors, 
and characteristics may be attributed to interviewer differences* 

Thus* variation in responses due to di fferences among interviewers may 
sometimes amount to a significant and important source of error in survey 
research* Moreover* such effects are likely to be particularly problematic if 
they tend to be correlated with certain basic characteristics of the respond- 
ent* such as age* sex* or education- With regard to age* there is indeed some 
research evidence which suggests that older people may be more susceptible to 
interviewer effects than younger people* For example, in various experimental 
situations older subjects are generally found to conform more readily to 
social influence manipulations and are more eager to please (Her2og» 1979; 
Klein* 1972; Klein and Birren, 1972)* In addition* they traditionally tend to 
score higher on measures of social desirability (Herzog and Rodger s» I98I; 
Campbell et a1«» 197 6) * It is quite possible that each of these basic tenden~ 
cies could result in increased susceptibility to influence by a survey inter- 
viewer * 

Despite the plausibility of this hypothesis, however* few studies of 
interviewer variance have even considered possible age differences in respond- 
ent susceptibility to interviewer effects, and those which have done so 
provide only indirect or limited evidence bearing on the issue* For example* 
Sudman and Bradburn's (197*0 comprehensive review of data on interactions 
between respondent and interviewer characteristics on response effects 
revealed no evidence of age effects for adult respondents* perhaps because the 
data available (e*g** Benney* et a1,» 1956* Ehrtich and Riesman, \?o\t Hanson 
and Harks* 195&) were too limited in age range to either prove or disprove an 
interaction effect due to respondent and interviewer age. Similarly, in two 
recent and more d i rect stud ios of 1 nterv i ewer var iab i 1 i ty* Ba i ley and his col- 
leagues (1978) examined the effects of National Crime Survey interviewers on 
survey statistics fry sex and race of respondents* but not by age; and Sudman 
et a1< (1977)* in spite of giving passing recognition to the possible 
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influence of age on interviewer contribution to total variance* neither con- 
trolled for respondent age (which was reported as not varying across inter- 
viewers) nor examined interviewer effects for possible age differences. To 
our knowledge* there have been only two studies of general interviewer effects 
which directly examined the relationship between respondent age and inter- 
viewer variance* One such study is that reported by Freeman and Butler 
(1976)* which* although it examined age of respondents only in combination 
with age of interviewers, found that "old 11 interviewers (32 and older) inter- 
v"* ing "old" respondents did indeed produce consistently more interviewer 
v t . ince than did any other age combination* The other such study was done by 
Groves and MagMavy (1980) , who found that interviewer effects were stronger 
for respondents age 60 or older than for young (age 18*2*0 respondents* for 
each of six types of questions that they examined* 



Data 

The major problem in the study of interviewer effects consists of 
separating such effects from true differences between groups of persons inter- 
viewed by different interviewers* In personal interview studies* it is 
generally not practical to assign interviewers to sampled households on a ran- 
dom basis* for national surveys in particular* the travel costs and delays 
Implied by such a procedure would be prohibitive* It is* however* possible to 
have "interpenetrating" designs* for example* one in which two or more inter- 
viewers are assigned randomly to sampled households within each geographic 
area* which permit estimation of interviewer effects (Mahalanobis* l9*+6; 
U*S* Census Bureau* 1979)* Estimation of interviewer effects is considerably 
more feasible with telephone Interviews* where typically all interviews are 
conducted from a central location* Of course* there is little basis for 
regarding estimates of interviewer effects obtained from a telephone survey to 
be unbfased estimates for interviewer effects in personal interview surveys; 
indeed* there is evidence that interviewer effects tend to be smaller in the 
telephone methodology (Groves and MagMavy, I98O) ♦ 

The data used in the analyses to be reported below were obtained in a 
stndy by Cannell and Groves (1979a)* using a computer-assisted telephone 
interviewing system* One feature of this system was that assignments of 
sampled telephone numbers to interviewers were generally made by the computer* 
initially on a random basis* then* after a number had been dialed without a 
response* in accordance with an algorithm that attempted to maximize the 
probability of a contact by taking account of the time of day and week* Since 
interviewers generally worked the same hours of day throughout the study* ran- 
domization could only be within each shift rather than across all inter- 
viewers* Moreover* some households that initially refused to participate in 
the study were then assigned to interviewers who had shown greater success in 
converting such refusals* but data from such interviews were deleted from our 
anal yses * 

The survey used a two-stage stratified sample of telephone numbers 
generated using the random digit dialing technique developed by Waksberg 
(1978)* In an effort to reduce nonresponse bias* a double sampling scheme was 
used for nonrespondents after the fifth week of the data collection* An 
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interview was conducted with one objectively selected adult in each sample 
household* A total of ]05U interviews were completed, which reoresent a 
response rate of 67 percent if all telephone numbers which were never answered 
are included in the denominator (an unknown proportion of such numbers are 
non-working< or not located in households). The thirty or so persons who con- 
ducted these interviews were selected from applicants for the job in Ann 
Arbor; none had previous interviewing experience with the Survey Research 
Center* but all completed a three week training period, Host were female (87 
percent), below thirty years of age (77 percent), and had completed at least 
two years of college (73 percent); about half were part time students (53 per- 
cent) . 



Methods 

Host studies of interviewer effects have utilized analysis of variance 
techniques, the independent variable being the interviewer identification, the 
dependent variables the responses to questions for which Interviewer effects 
are being measured. In our analyses, we employed a regression analysis 
approach; this provides somewhat different statistics than does the analysis 
of variance approach to the study of interviewer effects, so we will provide 
our rationale for using this technique and examine the comparability of our 
measures with those from an analysis of variance. 

The reason we chose to use regression analysis rather than analysis of 
variance is that regression analysis Is the more flexible technique, allowing 
greater complexity to be Incorporated more easily into the design matrix* 
Multiple and continuous variables (i*e*, variables with many categories) with 
linear relationships to th* dependent variable are easily included as predic- 
tors, whereas they are not easily handled (except as covariates) in analyses 
of variance* fn the analysis of Interviewer effects, for example, it might be 
interesting to use background characteristics of the respondents, such as 
education, age, and income, along with the interviewer dummy variables as 
predictors In order to examine interviewer characteristics as part of a 
broader explanatory model* In the analyses to be presented below, we included 
age and interviewer variables to examine their separate and interactive 
effects* The predictor variables in the regression analysis included a dummy 
variable for age (whether or not the respondent was 60 years old or older), a 
set of dummy variables representing all but one of the thirty interviewers, 
and another set of 29 dummy variables for the interaction of age and inter- 
viewers* This regression analysis Is formally equivalent to an analysis of 
variance with two independent variables: a two category age variable, and a 
thirty category interviewer variable* One reason for preferring the use of 
regression analysis even in this case, despite its formal equivalence to 
analysis of variance* is that the standard output from regression analysis 
includes a set of coefficients each of which represents the effect of one 
interviewer* These coefficients can be compared for different dependent vari- 
ables to determine whether certain interviewers consistently have large 
effects on the way respondents reply to their questions* 

To opeasure the magnitude of interviewer effects, we have used the 
adjusted R statistic, which is defined as: 
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R adj = 1 -[ (N "" /tM-c)] 
where N is the total sample size and c is the number of interviewers* 1 



F i ndi nqs 

Effect of interviewer and age of respondent 

The first question we address is whether interviewers tend to have the 
same effect on older respondents as they do on younger respondents* We used a 
set of items for which Groves and Magi tavy (1980) found significant overall 
Interviewer effects* Us ng an analysis of variance approach, these inves-* 
tigators tested a total of 7? items for interviewer effects* and found statis- 
tically significant effects (p<*05) for a total of 30 of these items* U of 
which are answers given by respondents* the other l6 observations made by the 
interviewers* We tested whether the age of the respondents interacted with 
the specific interviewers with respect to each of those 30 items* 

As noted before, in our regression analyses* dummy variables were 
included for all but one of the 30 interviewers involved in the data collec- 
tion* an analysis that is formally equivalent to an analysis of variance in 
which the 30 interviewers constitute categories on the single independent 
variable* F~tests for the overall significance of interviewer effects are 
equivalent for the two types of analysis* but our measure of the magnitude of 
interviewer effects differs somewhat from that used in the analyses reported 
by Groves and flagi tavy Os30:Tabte 1) * The results are reported in Table 8-1* 
The first column lists the 30 variables* the second column reproduces the roh 
statistic Reported by Groves and Magilavy* while the third column gives the 
adjusted R values from our regression analysis when only the interviewer durtr 
my variables are used as predictors* Comparing the two sets of val'jes in 
columns 2 and 3 # it is clear that they are very similar to one another* 

The last three columns of Table \ provide findings concerning the 
existence of interaction e f fects between interviewers and age of the respond* 
ents* Specifically* the fourth column lists the proportion of variance 
explained by interviewers and the dichotomous age variable, the "main effects" 
model* The next column lists the proportion of variance explained by a igodel 
that adds interaction terms for these two variables* (Note that these R 
values are adjusted for deg rees of f reedom , so that the va 1 ues i n the fifth 
column can ce f and sometimes are* smaller than those in the fourth column*) 
For eight of the thirty variables* there sre statistically significant inter- 
action effects* as shown by the F-statisttc listed in the last column* These 
tests of significance should be regarded with some caution, since there is 
evidence that the variability of answers obtained by some interviewers is con* 
siderably larger than those obtained by other interviewers, a violation of the 
assumption of homoscedasti c i ty which underlies the F-test* Nevertheless, it 



1 A statistic that has more frequently been used to measure interviewer 
effects Is the "ratio of homogeneity*" roh* As shown in £he Appendix* if the 
number of interviewers c t is fairly large* both roh and R a(J , must both lie 
within narrow bounds* and thus must have similar values* 
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does appear that for a substantial proportion of variables, among those for 
which there are significant interviewer effects* there are also interactions 
between interviewers and the age of the respondents* in terms of explanatory 
power* comparison of the fourth and fifth columns of Table 1 indicates that 
the interaction effect explains between 1*5 and *+*5 percent of the variance 
beyond that explained by the main effects of age and interviewers* 

As noted before, the last 16 variables listed in Table 1 ?re observa- 
tions made by the interviewers after the completion of the interviews rather 
than responses given directly by the respondents* Of the 77 items that Groves 
and Magilavy examined for interviewer effects 19 were interviewer observations 
and 58 were answers by respondents to questions* Clearly, the interviewer 
observations were much more likely to be subject to interviewer effects than 
were the answers given by respondents; 16 out of the 19 observations were 
found to have statistically significant interviewer effects, compared to 14 
out of the 58 responses. However, as shown in the analyses reported here, 
these interviewer observations were no more likely to show significant age by 
interviewer interaction effects than were t:ie responses given directly by the 
respondents: 5 of the }U responses have statistically significant interaction 
effects, as compared to 3 °f the 16 interviewer observations, as shown in 
Table 1* We conclude that interviewers are more likely to differ systemati- 
cally in the way they record their own observations than in the way they 
affect the answers given to them by respondents; but that age-by-interviewer 
interactions seem no more likely in the case of these interviewer observations 
than in the case of answers given by respondents* 

Groves and Mogilavy also reported that responses to open-ended ques- 
tions tended to be subject to considerably larger interviewer effects than 
responses to closed-ended questions* If anything* the opposite pattern may 
hold with respect to age-by-interviewer interaction effects* however* The 
first six items listed in Table 1 are closed-ended* and for three of them 
there is a statistically significant interaction effect, whereas only two out 
of the eight open-ended questions showed significant interaction effects* In 
short* there is no evidence that age-by-interviewer interactions are more 
likely for open-ended than for closed-ended questions* 

The analysis just described tells us that for a substantial proportion 
of variables which have significant interviewer effects* there are also sig- 
nificant interactions of interviewers and the age of the respondents* but it 
'.ells us nothing about the nature of those interactions* tn particular* it 
does not address the primary question which motivated our investigation: 
whether interviewers tend to have more effect on answers given by older 
respondents than on those given by younger persons* To answer this question 
directly* we performed parallel regression analysis on two groups of respond- 
ents; those under the age of 60* and those age 60 or older* The magnitude of 
interviewer effects were estimated for each of the 30 variables within each 
age group. These magnitude estimates were described in the preceding section, 
and are adjusted to take account of the degrees of freedom and the size of the 
sample* thus the adjustments are considerably large** for the older group than 
for the younger group* since there were only 158 respondents age 60 or older 
compared to 785 under the age of 60* 

The interviewer effects on each of the 30 variables and for each age 
group are shown in Table 2, The average interviewer effect (i*e** adjusted 
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R ) for those age 60 or older is 7.5 percent* which is slightly higher than 
the average effect for younger respondents (7*0 percent) * Since the distribu- 
tion of values of the adjusted R values are considerably skewed to the right* 
a more meaningful measure of the central values may be the medians: 6.2 per- 
cent for older respondents* which is twice as large as the median of 3*1 per- 
cent for younger respondents* As noted earlier, most of the variables for 
which there are large interviewer effects are interviewer observations rather 
than questions answered by the respondents. If we look only at the questions 
that were asked of respondents (i*e*, the first \U items in Table 2)* the 
median proportion of explained variance is 5*0 percent for the older respond- 
ents* which is more than twice as large as the median of 2*0 percent observed 
for younger respondents* It appears that the proportion of variance in 
answers given by older respondents is greater than in the case of younger 
respondents* On the other h.ind, for the 16 interviewer observations listed at 
the end of Table 2 there is no evidence that t^ere are stronger interviewer 
effects for older than for younger respondents: the median percent of 
explained variance is 8*0 percent for older respondents, which is actually 
1 ess than the median of 10*8 percent for younger respondents* and for only 7 
of the 16 items is the explained variance larger for the older than for the 
younger respondents* This observed difference for the two types of questions 
seems intuitively reasonable* since interviewer ratings are more exclusively 
under the control of the interviewer* while answers by respondents are a 
result of the interaction between the respondent and th± interviewer* 



Checks on the Generality of the Findings 

Importance of "Outlying" Interviewers , It is important to know whether 
the interviewer effects that w* have observed reflect a small proportion of 
interviewers who rather consistently obtain answers from respondents that dif- 
fer markedly from those obtained by other interviewers* or whether they 
represent a "normal" range of effects due to all of the interviewers* If a 
minority of interviewers were responsible! it might be possible to identify 
them and either retrain them or remove them from the interviewing staff* If 
some interviewers have relatively strong effects on certain types of questions 
(but not on others), it might be possible to identify interviewer characteris- 
tics that are associated with positive or negative biases on spec i fi c* ques- 
tions and tfc correct for those biases in the analysis* 

Our first approach to this question was to examine the distributions of 
t'*e regression coefficients 0*e. # the deviations of the mean response 
obtained by each interviewer from the overall mean response to a question) 
estimated for each of 13 dependent variables* These variables represented 
answers by respondents rather than interviewer observations. The coefficients 
for each of these variables were examined for each of the 28 interviewers who 
interviewed both younger and older respondents* for a total of coeffi- 
cients for each age group* 

Almost all of the coefficients with large absolute values (specifical- 
ly* these that deviated from the overall mean for a dependent variable by at 
'east two standard deviations) were associated with a few interviewers who 
•V ned only a small number of interviews* The estimated effects of such 
viewers are obviously less precise than for more productive interviewers; 
*.ms shows up in the form of larger standard errors for the regression coeffi- 
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cients corresponding to those interviewers in the regression analysis* To 
determine whether this set of interviewers wa responsible for the significant ^ 
interviewer effects aiid/or for the significant age-by-interviewer interaction 
effects* we repeated these regression analyses omitting the dummy variables 
associated with all interviewers who failed to obtain at least five interviews 
with persons under the age of 60 and at least three interviews with older 
persons — a total of eight out of the thirty interviewers* For this analysis, 
we considered only the set of questions asked of respondents for which Groves 
and flagilavy found statistically significant interviewer effects--the first 
fourteen variables listed in Table 1. 

These alterations did affect the findings* but not in what we consider 
a major way* 2 Our conclusion Is that while some of the variance attributed 
to interviewers reflects the influence of a small number of interviewers who 
interviewed only a few respondents* a substantial amount of Interviewer 
variance is due to a fairly normal distribution of effects associated with 
individual interviewers* 

Consistency of Interviewer Effects across Questions * We next asked 
whether the effects attributed to individual interviewers are consistent 
across questions; that is, whether certain interviewers tend to bias answers 
given to them in,a particular direction* This analysis was done for all 
interviewers and for all thirty of the items listed in Table 8-1* but for 
simplicity we only report our findings from an analysis restricted to items 
asked of respondents (omitting interviewer observations) and to Interviewers 
who took at least a few interviews with both younger and older respondent* 
(omitting the eight interviewers described in the preceding section), 

The average correlation of the interviewer effects on responses to dif- 
ferent questions among younger respondents was 0,21*2; for older respondents* 
the average was 0,06$, For some of the items the interviewer effects are 
rather substantially correlated. Cluster analysis was used to identify a sub- 
set of nine items with respect to which the average correlation of the inter- 
viewer coefficients for younger respondents is 0 • 3^6 * 3 This suggests that 
there may indeed be a consistent tendency for interviewers to obtain answers 
that are consistently biased in one direction or another. The correlations of 



a The overall proportion of variance explained by interviewers (the 
average of the adjusted R~square values) dropped by about a third* from 2*U to 
1*6 percent* and interviewer effects were no longer significant at the ,05 
level for three of the fourteen variables, Statistically significant (p<,05) 
aoe-by- i ntervi ewer interaction effects were detected for four of the fourteen 
variables; two that are noted as being significant in Table 1 (V1013 and 
V 10 1 6) are no longer significant at this level when the eight interviewers are 
omitted* while a statistically significant (p<,05) interaction is now found 
for one variable (V1005) for which none was detected when all interviewers 
were considered, Moreover* inspection of the estimated regression coeffi- 
cients revealed only *i,3 percent that differed by two or more standard devia- 
tions from the unweighted mean across all of the remaining interviewers, 

*These nine items are* using the variable numbers listed in Table 1* 
^815- V921+, V926, V928* V1005, V1025. V875> V1013* and V1021+, We have not, 
however* tried to attach any meaning to this cluster of items. 
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the coefficients for the older respondents are considerably lower, however* 
perhaps because the number of older respondents in this study was rather small 
thereby yielding unreliable estimates of the effects of individual inter- 
viewers, ft is also interesting that there is essentially no relationship 
between the relative effect of an interviewer on younger respondents and his 
or her effect on older respondents; the average correlation of the estimated 
effects across the items is only 0.089* For some of the items* the correla- 
tions are in the range from ,1+0 to .50, but for other items the correlations 
are actually negative** 



Using data collected through telephone interviews with a national 
sample of adults, we have searched for evidence as to whether or not inter- 
viewers have stronger effects on the responses given to a wide range of ques- 
tions by older people than on the responses of younger people. Specifically* 
we examined responses to a set of thirty items for which significant inter- 
viewer effects had earlier been reported, out of a total of 77 items that were 
screened for such effects* 

Significant age-by- i nterviewer interactions were detected for eight of 
the 30 items. The number of i tercs is too small to permit conclusions to be 
drawn with any certainty about types of items that are more or less likely to 
be subject to such interactive effects* The set of eight items for which sig- 
nificant interactions were detected in this data set included both interviewer 
observations and the direct responses of interviewees, and among the latter 
responses to both open- and closed-ended questions. 

We next examined older and younger respondents (i.e.* those age 60 and 
older vs. all others) separately to determine the nature of the age-by- 
interviewer interactions* No pattern could be found for the interviewer 
observations, but for the direct responses to questions the median interviewer 
effect was more than twice as strong for older respondents as for younger 
respondents . 

Some of the interviewer effects that we observed can be attributed to a 
small number of "deviant 11 interviewers who obtained answers quite different 
from those obtained by most other interviewers; these interviewers usually 
obtained only a small number of completed interviews* so that our estimates of 
their effects on the respondents are less reliable than for those who were 
more productive. However, removing t*hese interviewers from the analysis did 
not change the basic pattern of the findings* We conclude that the observed 
age-by-interviewer effects probably reflect a range of interviewer charac- 
teristics and behaviors rather than a minority of poorly trained or deviant 
interviewers. With the present data* however, we can say nothing about what 
might be the nature of such characteristics* 



'Variables numbered V8)5 (r = .1+1), V950 (r ■ .1+1), V1025 {r = .1+0), and 
V102i* (r - .1+9) . 

^Variables numbered V926 (r * -.07)* VS28 (r = -.08), V9*+6 (r = -*03) * 
and V1005 (r = -.09) * 



Summary and Conclusions 
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We also looked *or evidence of consistency of the patterns of inter- 
viewer effects across different questions* For a subset of the questions 
answered by younger respondents we did find some evidence for at least a small 
degree of consistency, but no such consistency was found in the effects on 
older respondents* Moreover, the estimates of the effects of an interviewer 
on the answers given by younger respondents to a particular question were 
essentially unrelated to the estimates of the effects of that interviewer on 
the answers to the same question given by older respondents, 

Based on the evidence that we have presented, it appears that older 
respondents may be somewhat more susceptible to interviewer effects than 
younger respondents, This implies that careful training of the interviewing 
staff is even more important when the target population is elderly than is 
true for genera 1 popul at i on surveys * Techniques for i ntroduc i ng standard i zed 
methods of asking questions and probing for clarification or amplification of 
answers should be implemented whenever possible, and in perticuler when th e 
is an emphasis on older persons. Procedures of instruction, commitment, and 
feedback developed by Cannell et al* ( 1 98 1 > and described elsewhere in this 
report have had the effect of reducing interviewer variance and might 
therefore be advisable for interviews with older adults. 

Another implication of these findings is that the effective size of 
samples of older respondents may be considerably smaller than the apparent 
sample size. Interviewer effects, in a manner analogous to the more familiar 
design effects associated with complex samol ing frames* have the consequence 
of increasing the magnitude of the errors associated with any sample statistic 
and thereby of reducing the effective sample size, This Implies, for example, 
that the nominal probability levels cited for tests of hypotheses may be too 
lenient, perhaps leading to acceptance of hypotheses that are incorrect and 
thus, over the long run, to more frequent failures of attempts to replicate 
findings from previous studies. 

These conclusions should be qualified* since our analyses of interviewer 
effects were based on data collected through telephone interviewing and we 
cannot be certain that similar patterns would be found in data collected by 
face-to-face interviews. An important difference between the typical 
telephone survey and the typical face-to-face survey is the number of inter- 
views obtained by each interviewer; telephone interviewers* not restricted by 
geography, often collect several times as many respondents as do face-to-face 
interviewers who are confined to a small geographic area* Since the magnitude 
of the interviewer effects is proportional to the average number of respond- 
ents per interviewer, we would expect that if the average magnitude of the 
effect of an individual interviewer were about the same for the two interview* 
ing modes, the overall interviewer effect would be considerably larger for 
telephone interview surveys than for face-to~face surveys, 

On the other hand, there is little basis for thinking that individual 
interviewers would have any less influence on the respondents when they are 
face to face* and therefore perhaps more likely to communicate by non-verbal 
cues* than when they are i n contact onl y by a te 1 ephone 1 i ne , 1 ndeed, Col 1 i ns 
(personal communication, 1982) analyzed data from a face^to-face interview 
survey conducted in England, and found a similar pattern of interviewer~by-age 
of respondent interactions to those that we have discussed here, Moreover, 
the data used in our analysis were collected by a staff of interviewers in a 
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central location* where they were monitored throughout the data collection 
period* whereas the field staff in a face-to-face interview survey may be 
scattered over the entire country and feedback on their performance from the 
research staff is often necessarily very slow if not entirely lacking* For 
this reason we suspect that there is probably greater uniformity in the tech- 
niques and styles used by telephone interviewers than is typically true of a 
staff of face-ta-face interviewers, and thfs leads us to expect that inter- 
viewer effects are even stronger for face~to~face interview surveys than those 
we found for this telephone interview study. Whether interviewer-by-age of 
respondent interactions may also be more frequent in face-to-face interview 
studies than we observed in this study would be purely speculative; but we see 
little basis for expecting such interactions would be any less frequent in 
face- to- face interviews than in telephone interviews* 

Pending the collection of more data that would permit the direct evalua" 
tion of the frequency and magnitude of interviewer effects on older respond- 
ents! our conclusion is that caution should be applied in the interpretation 
of data obtained by f ace~to~f ace interviews with the elderly* Interviewer 
effects may be substantial* and could distort the apparent probabilities of 
sample statistics; and could introduce biases into such statistics, since 
there is no guarantee that the average effect across a set of interviewers is 
zero* This caution should be particularly heeded when the number of respond- 
ents interviewed by each interviewer is substantial. 
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APPENDIX 



The "ratio of homogeneity." or roh, which is defined (cf. Kish. 
1965: 172) as fol lows: 

2 

where s is the "between interviewer 11 component of the variances: 
a 

s 2 = [l/<a-l)] I (y. - y) 2 
a j = 1 1 

= SSR/[b(a-l)] - (SSTO - SSE) / [b (a- 1) ] ; 

2 

s^ is the "within interviewer 1 * component of the variance; 

s 2 - 1/[a(b-l)] I I (y M - y.) 2 
D i = 1 J= 1 J 1 

= SSE/[a(b-l)]; and 

s 2 = s 2 + [(b-l)/b]s 2 

= SST0/[b(a-1)] - SSE/[ab(a-l)]i 

with a being the nu.nber of interviewers and b being the number of Interviews 
taken by each interviewer** In our regression analyses* the measure of inter- 
viewer effects is the adjusted R value, defined as follows: 

R adj = 1 " C(N-D/ (N-c)] (1 R 2 ), 
This can be rewritten as follows (since N = ab and c ■ a) : 

2 

both roh a &nd the adjusted R~ value must 



adjusted 
1 - [b/(b»t)]SSE/SSTD £ 



shows us that 
lowing bounds: 



R 7 « 1 - [(ab-l)/{|b-a)]SSE/SSTO* A little algebra 
h and the adjusted R value must lie within the folio 

£ 1 - C(a-l)/a] * [b/(b-l)]SSE/SSTO, 



roh 
2 
adj 



*ln practice* the number of interviews taken by each interviewer is not a 
constant; In the present survey t one of the 30 interviewers took only U inter- 
views, while another took more than 80 interviews* The expression for roh 
should be modified in terms of the average number of interviews per inter" 
viewer; see Groves and Hagilavy O980) ♦ The sampling distribution of roh is 
also effected by variability in numbers of interviews per interviewer* 
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Table 1: Interviewer. Age, and Interaction Effects on 
Selected variables from the Groves-Cannel 1 Telephone Survey 





Variable Name 


roh 




R' adj . 




F test 
i Interact Ion J 


INT 


INT , AGE 


I NT x AGE 


V8 15 


I watch television because there 
is nothing eis*» to do: hfcrdly ever 


.018 


.0170 


.0227 


.0178 


0.836 


V924 
V926 
V928 


Proport Ion ever" having had' 
Arthritis or rhev.nM Ism 
Heart disease or any heart trouble 
Hypertension or high blood pressure 


.023 
.021 
.018 


.0218 
0201 
.0167 


. 1202 
.0438 
.0689 


. 1487 
.0850 
,091 1 


2.100* 
2.475* 
1 . 803+ 


V932 


Proportion having made contributions 4 
to the American Heart Association 


.022 


.0210 


.0523 


.0544 


1.069 


V94G 


tn the previous month drank hard 
liquor: proportion not at all 


.016 


.0145 


.0224 


.0119 


0.646 


V947 


Time elapsed since last doctor's 
visit, tess than 6 months 


.027 


.0257 


.0294 


.0330 


1.115 


V950 


Proportion able to report tfato 
of most recent doctor's* v1<?it, 
precise within a week 


.03 1 


.0306 


.0299 


.0248 


0.836 


V1O05 


Number of health conditions 
reported in open questions 


,016 


.0152 


.0467 


.0586 


1.418 


VI 025 


Number of ameliorative health 
behaviors reported 


.028 


.0271 


.0348 


.0327 


0.930 


V875 


dumber of ways mentioned that TV 
ts good or bad fo^ children 


.016 


.0146 


.0163 


.0128 


0.887 


V1013 


Length of response for mentions 
of "bad reactions** to medicine 


.024 


.0230 


.0244 


.0425 


1.621* 


V1016 


Length of response for mentions 
of health symptoms 


.031 


.0298 


.0318 


.0520 


1 .698* 


VI 024 


Length of response for mentions 
of ?me1 iorat 1v€> heal th behaviors 

Interviewer Observat ions : 


. 06 1 


. 058 1 


. 065 1 




v. ov4 


V902 


Proportion of -espondents "never" 
giving an answer so long that the 
Interview was delayed by typing 


.241 


.2329 


.2324 


.2385 


1.253 


V909 


Proportion of interviewers having 

a problem with the terminal control keys 


.047 


.0446 


.0449 


.0785 


2. 168# 
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Var a 



Variable Name 



V912 Proportion of Interviewers reporting 
that there were **somet imes** slow 
system response times 

Did respondent give an Indication of 
checking any records to look up 
last doctor's visit? 
VS03 Proportion uncertain 

VS04 Proportion no 



VS7S Proportion of respondents "not at all" 
reluctant to agree to interview 

VB8 1 Proportion of respondents "not av alt" 
susP I c ious about survey 

V8S2 Proportion oF respondents whose 
suspicion was reduced as the 
interview progressed 



VS85 



Proportion of respondents who 
seemed to rush their answers 



VSS6 Proportion of respondents who did not 
ask how much longer the interview 
would last 

VS90 Proportion of respondents who seemed ^ 
-somewhat interested in the interview 

VS92 Proportion OF respondents who "never" 

asked for clarification during the interview 

VS9G Proportion of respondents who 

needed **a Few question** repeated 

V8D8 PrOpOr t ion oF respondent s suspected 
of not giving itonest answers 

V903 Proportion OF respondents making 

"very much effort** to give complete 
and accurate answers 

V90G Proportion oF respondents H very likely" 
to grant a reHnterview 



* p < .05 



* p < .01 



p < .001 



9 

ERLC 



R* adj . 

roh F test 

INT INT* AGE iMTxAGE ( interact ion) 



. 1C4 .0998 .0988 .0991 1.011 

.237 .2295 .2288 .2145 0.399 

. 165 . 1593 ■ 1584 . 1605 1 ,083 

.055 .0537 .0681 .0735 1 . 123 

.087 .0834 .0965 .1125 1.579* 

.180 . 17 10 . 1871 . 1658 0*785 

. 025 . 0237 . 0234 . 03 1 2 1 . 258 

.023 .0223 .0548 .0688 1 *4S2 

.036 .0342 .0334 .0424 1.299 

. 186 . 1794 . 1804 . 1873 1 .273 

.092 .0869 .0885 . 1333 2.657# 

.021 .0198 .0205 .0088 0.621 

.152 . 1462 . 1454 . 1451 0.988 

.096 .0930 .0990 - 1073 1.295 
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Table 2: interviewer Effects on Younger 
(Under age 60) and Older Respondents 



Vnr - M 


Variable Name 




R' adj. 




<Age < 60) 


<Age > 60) 












V815 


I watch television because there 

■ 9 llv III II ly | o-tj » W QU v "PI Ml y ?VCI 




.0049 


.0901 


V926 
V928 


Proportion ever having had; 

Ap thr i ♦ i ■■c nr rhAiiMA ♦ f on 

f 4 1 llll 1 l r9 wi 1 * ICUIIIO \ r *MII 

Heart disease or any heart trouble 
Hypertension or high blood Pressure 




.0108 
.0186 
.0212 


.0547 
.0679 
.0304 


V932 


Proportion having made contributions 
to the American Heart Association 




.0169 


.0321 


V946 


Tn thp DrAuf nu^ month drank hard 

J.J4 111? pr| CV 1 UU9 IIIUI 1 \ 1 t VI * f Ul %f 

liquor: proportion not at all 




.0123 


(-.0428) 


V9.17 


Tinp elapsed titnrp la*it doctor's 
visit* less than 6 months 




.0219 


*0543 


v a iiv 


of most recent doctor's visit* 
precise within a week 




.0220 


.0463 


V1005 


Number of health conditions 
reported in open questions 




.0163 


(-.0072) 


V1025 


Number of ameliorative health 
behaviors reported 




.0184 


{- 00i3) 


V875 


Number of ways mentioned that TV 
is good or bad for children 




.0247 


(-.0892) 


V1013 


Length of response for mentions 
ftf "luirf rAftrt (nnc* to medicine 




.0228 


. 1033 


V1016 


Length of response for mentions 
of health symptoms 




.0431 


.0492 . 


Vi024 


Length of response for mentions 
of ameliorative health behaviors 

Interviewer Observations: 




.0525 


.0387 


V902 


Proportion of respondents "never" 
0'Vi"»g an answer so long that the 
interview was delayed by typing 




.2475 


, 1793 


V909 


Proportion of interviewers having 
a problem with the tr^minal control 


keys 


,0538 


,4781 
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Var r if 



V912 



Var iabie Name 



R' adj 



(Age < 60) 



(Age > 60) 



V803 
V804 

V878 



V881 
VS82 

V885 
V886 

V890 
V892 
V896 
V898 
V903 

V906 



Propor t ion of Interviewers repor t ing 
that there were "sometimes - slow 
system response times 

Did respondent give an indication of 
checking any records to look up 
last doctor's visit? 
Proportion uncertain 
Proportion no 

Proportion of respondents *not at all* 
reluctant to agree to interview 

Proportion of respondents *not at all - 
suspicfous about survey 

Proportion of respondents whose 
suspicion was reduced the 
interview Progressed 

Proportion of respondents who 
seemed to rush their answers 

Proportion of respondents who did nox 
ask how much longer the interview 
would last 

Proportion of respondents who seemed 
"somewhat" interested In the interview 

Proportion of respondents who "never" 

asked for clarification during the interview 

Proportion of respondents who 
needed *a few qv^stion" repeated 

Proportion of respondents suspected 
of not giving honest answers 

Proportion o* respondents making 

"very much effort" to give complete 
and accurate answers 

Proportion of respondents *very likely" 
to grant a re-Interview 



1122 



,2002 
.1445 



.0342 



.0983 



.2392 



.0284 



.0269 



.0415 



, 1602 



. 1318 



.0163 



1483 



. 1033 



.0228 



,2916 
.2423 



.0993 
.0827 

(-. 1584) 
.0222 

(-.0297) 
.0381 
.3578 
. 1007 

(-.0809) 

*067 
.0632 
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